An Automatic Algorithm Designer: An Initial Implementation

This paper outlines a specification for an algorithm-design system (based on previous work involving protocol analysis) and describes an implementation of the specification that is a combination frame and production system. In the implementation, design occurs in two problem spaces one about algorithms and one about the task domain. The partially worked out algorithms are represented as configurations of data-flow components. A small number of general-purpose operators construct and modify the representations. These operators are adapted to different situations by instantiation and means-ends analysis rules. The data-flow space also includes symbolic and test-case execution rules that drive the component-refinement process by exposing both problems and opportunities. A domain space about geometric images supports test-case execution, domain-specific problem solving, recognition and discovery. This research is supported by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 3597, monitored by the Air Force Avionics Laboratory Under Contract F33615-81 -K-1539. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the U.S. Government UNIVERSITY LIBRARIES . CARNEGIE-MELLON UNIVERSITY • PITTSBURGH, PENNSYLVANIA 15213 Kant&Neweil An Automatic Algorithm Designer 1 1. Approaches to Design We are interested in systems that automatically design algorithms and create programs for them. The process of algorithm design requires a significant degree of both knowledge and intelligence, and may even require additional structure to make creative discoveries. In this paper, we briefly present our specifications for a design system and describe an initial implementation. The main goal of the research is to produce a successful automatic design system, but a secondary goal is to show that careful examination of human behavior can suggest novel and worthwhile system organizations. Related research on program synthesis and transformation, formal derivation, and automated discovery suggests some possible approaches to the automation problem. However, none provides a satisfactory paradigm for the whole task of automatic algorithm design. Program synthesis systems [1,5,13] often successively refine programs by transformations. Most of these systems lack robustness because they require numerous transformation rules to specify all the details about programming and because they have no problem-solving abilities other than simple pattern matching on the rules. This may indicate the desirability of having a few general operators with more sophisticated techniques for adapting rules to situations. Formal derivation systems typically do have a small number of general operators. However, they require complete, formal specifications, which may not be available initially [14], and they often apply rather rigid methods to choose among applicable transformation rules. People still must specify the interesting lemmas or auxiliary definitions and decide among alternative axiom sets. One system [2] does address the choice problem by guessing recursive solutions to logical equations and verifying the guesses with a theorem prover and a small model. Discovery systems are quite rare, and those that exist are either open-ended exploration systems rather than problem-solving systems that focus on a specific task [3,10] or work in very narrow domains such as algebraic models [9]. We propose an approach that includes relatively few basic operators, specific knowledge about instantiation rather than many transformation rules, and several problem-solving methods that work with domain knowledge. We also attempt a structure that permits discoveries. 2. A Functional Specification for an Algorithm-Design System The specifications for an algorithm-design system described here are based closely on the results of protocol analysis [6,7]. We take algorithms about computational geometry (for example, producing the convex hull of a set of points in the plane) as our first task domain because it forces us to consider the interplay between external geometric models, mental imagery, and computationally efficient serial algorithms. We analyzed protocols of human problem-solving and design strategies in algorithm-discovery tasks because a system organization that parallels human reasoning lets us draw on human experts for techniques, promises to be flexible and robust, and has potential for learning. KantSNewell An Automatic Algorithm Designer 2 2 . 1 . Algorithm-design methods to be supported Design begins with the hypothesis of a kernel schema or solution plan and proceeds by successive refinement. Two closely related methods of program execution guide the gradual refinement of the initial schema. Symboiic execution uncovers information by running a partial algorithm description with symbols as data. During execution, the algorithm and symbols are elaborated (new assertions and new components are added). If symbolic execution is not sufficient to permit progress, a specific example is constructed that permits detailed execution (test-case execution). In addition to driving the successive refinement, the two execution methods generate consequences and expose problems and complications. The test-case execution method allows the problem solver to generate examples that contain the right level of detail to make progress (the relevant knowledge must be evoked by concrete retrieval cues) while avoiding complications (such as boundary conditions) that are not important until the basic outline of the algorithm has been developed. However, test-case execution is more time consuming than symbolic execution because loops are executed for individual items, whereas symbolic execution is linear in the size of the algorithm structure. Thus, if the system understands a step well enough, it will execute it symbolically. Simple inferences may also be made during the execution process. Design includes problem solving and reasoning about the task domain. Examples are generating test cases, attempting to find counterexamples, and "proving conjectures by demonstration (first on "typical" and later on boundary-condition examples). An extreme, but important, example of reasoning is discovery, the sudden viewing of a situation from a new perspective that presents an opportunity to solve an outstanding problem or make an improvement What is perceived is not a solution to the main goal at the time the insight occurs; however, there is usually some preparation for the discovery some previously considered but unsolved problem. The new perspective often arises out of a novel and accidental alignment of objects or facts. A general recognition-based control structure for all the methods seems to be the appropriate device to support the possibility of such juxtapositions. 2.2. A problem space for representing algorithms Much of the design process occurs within a single problem space of schematic structures that represent algorithms. A small number of basic types of components are instantiated and configured in multiple ways to represent algorithms. Expert design concepts (for example, schemas for divide and conquer and for dynamic programming) are built up from and coexist with the simpler vocabulary. Components may be augmented with assertions that give additional information such as an ordering for a generator or a predicate on a test An assertion is not limited to concepts in the algorithm space. For example, it can describe a relationship in the task space that must be satisfied. Our analysis of human protocols showed that half of the design time was spent on symbolic and test-case execution, and design time was proportional to the number of components in the algorithm description. Even when subjects had difficulties in designing the algorithms, their unsuccessful behavior had a similar character of hypothesizing new structures and repeatedly attempting to execute thfe partial algorithm on sample data. Symbolic and test-case execution also had major roles in verifying, describing and analyzing algorithms. • — Our human subjects sometimes made discoveries while feeling lost and staring "blankly" at a figure during test-case execution. . Kant£ Newell An Automatic Algorithm Designer 3 The assertions drive the decisions about how to refine a component into a new configuration of components. The problem space must include operators that refine partially specified structures (add new components, links between components, or assertions) and that carry out the execution methods. The refinement operators must not insist that configurations form complete or consistent algorithms. While most components will have a set of standard inputs and outputs to represent primary data relationships, operators can add new connections at any time without checking for type mismatches. Instead, problems noticed during test-case or symbolic execution will be handled as they arise. All the problem-space operators should be very general and will require instantiation before they are applied. Since kernel ideas, instantiation, and indeed all design decisions can be wrong, design involves search in the problem space. Most search-control knowledge should be provided by rules that select instantiations of the operators that modify components and assertions and select parts of the algorithm to refine or execute. In the absence of specific knowledge for instantiation, other searchcontrol rules must provide more general techniques, such as means-ends analysts, to help adapt an operator to a specific problem state. 2.3. A problem space for the application-domain The application-domain space must support test-case execution of algorithms, problem solving about the domain, and discovery. Test-case execution requires operations that produce specific examples (and judge their suitability) and operations that evaluate assertions (such as predicates on tests) in the domain space. Our initial task domain will be a problem space for manipulating geometric images. It includes objects, such as points, line segments, and polygons; specific geometric operators, such as create a point or construct a line segment; and general operators (perceive, find, partition) that are typically specialized in geometric ways (partition a point set). The operators help create test cases (such as sets of points) and check assertions (such as whether a point lies above the X-axis). Working within a geometric task domain imposes important (but largely unknown) constraints, as indicated by the facility with which humans process visual scenes and reason spatially. Our design for a geometric problem space is modeled upon the imagery simulation of Kosslyn and Shwartz [8]. It consists of a short-term image buffer (a planar array), with the image centered around a focal point of attention. External (perceived) and internal (imagined) images are coalesced into a single image. Several operators (rotate, pan, zoom and scan) change the focus by regenerating the buffer around the focus point, rather than shifting the center in a fixed space. A variable amount of detail is present depending on point of view created by the movement operators. The image is viewed by a recognizer centered on the focus point, with the image memory permitting recognition of combined scenes. However, there are no operators that construct the image directly (write new patches of image into the buffer). Rather, all parts of the image are supported by the underlying semantic representation, and modifications are made to this underlying structure which is then pictured in the image. Discovery seems to occur through a recognition of some interesting configuration in the task space. Certain configurations of data items lead to recognition or automatic inferences (for example, completing polygons that are missing one edge, seeing polygons in regular patterns of points, being KaniSMewell An Automatic Algorithm Designer 4 reminded of related algorithms). The image structure just described seems to be appropriate to facilitate this recognition. 3. An Implementation of an Algorithm-Design System We are implementing an algorithm-design system that is both a protocol analysis/simulation system and a fully automated design system. The basic structure is a combination frame and production system. It is implemented in Lisp and can run with or without the OPS5 production system [4] as the automated control structure. In the simulation mode, the user (rather than the production rules) makes the decisions about which design commands to invoke and how to instantiate them. Design commands include component, assertion, and item construction, symbolic and test-case execution, and goal creation and modification. The implementation also includes objects such as protocol phrases, episode groupings, and comments (all represented by frames), and it has operators that create and edit the objects and insert them into a history of design commands. The total history describes an actual protocol and its interpretation. The user interface includes facilities for graphic display of componentconfigurations, fqr saving-, restoring, and undoing history sequences, and for command completion (with prompts for arguments and access to help files). The automatic mode is an augmentation of the simulation mode. Normally, the production rules make instantiation and search-control decisions and a history of design steps only (no protocols) is maintained. However, the modes can be mixed and the user can take over or relinquish control in midstream. 3 . 1 . Representing algorithms with data-flow configurations Partially specified algorithms are represented as states in a data-flow problem space, DFS. Each state is a data-flow configuration. The algorithm steps are represented by process components. There are a small number of generic process components (a memory, and the flow-control components generate, test, and select) and a general apply component that can be specialized to domain operations such as make-line-segment The inputs and outputs of the process components are represented by ports connected by links. Process components can be further specified by assertions. The components and assertions together modify and control the flow of items that represent data objects such as points and line segments. Items can fill several roles •• generic descriptions, symbols (for symbolic execution), and specific domain-space data. Assertions can be attached to any object, including an item, a process component, a whole configuration of components, and a problem space. Figure 3-1 shows an example of a simple OFS configuration that, given a set of points, finds the subset that lies above (on the left side of) the X-axis. The input, {i}, is a memory containing a set of points, { A B C D}, which are enumerated by Generate1 . These points that satisfy the predicate associated with Test., are added to the output set memory {o}. In the figure, points A, B and C have been enumerated. A was discarded, B has been added to the output, and C has not yet been tested. Data-flow representations have been studied in computer architecture research and have been used to describe artificial intelligence programs [11,12] and to express algorithm transformations [15]. Kant&Newell An Automatic Algorithm Designer